Soccer-specific skill tests are contemplated as objective and reliable measures of skill proficiency (see comprehensive reviews (Ali, 2011; Russell and Kingsley, 2011)), while being able to discriminate young soccer players according to playing position, current level or future success (Coelho e Silva et al., 2010; Huijgen et al., 2009; Reilly et al., 2000; Vaeyens et al., 2006). Recently, dribbling speed, shooting accuracy and passing tests were acknowledged as important discriminating factors between goalkeepers and outfield players, while fat-free mass and ball control entered the model in middle and late adolescence, respectively (Rebelo-Goncalves et al., 2015). Available data comparing goalkeepers of different playing standards (Gil et al., 2007; le Gall et al., 2010) are limited to anthropometric and physiological characteristics with the exception of one study (Rebelo et al., 2013), where elite U-19 goalkeepers were largely differentiated from their non-elite peers in ball control skill. However, the distinctive technical demands of goalkeeping claim the need for test measures regarding the performance-related characterization of soccer goalkeepers (Ziv and Lidor, 2011).

While their movement patterns are mainly characterized by long periods of low intensity, soccer goalkeepers are required to perform moderate-high intensity multi-directional movements and a number of skilful actions. Accordingly, the average number of sprint actions was reported to be 2 ± 2 with a total distance range between 0 and 15 m, with a higher prevalence of sprints of 0 – 5 m reported in sixty-two goalkeepers from 28 teams in the English Premier League (Di Salvo et al., 2008). One of the most critical movements involved in goalkeeping is the diving action. During the 2002 FIFA World Cup in Korea and Japan (De Baranda et al., 2008), goalkeepers performed up to 17 dives (6.2 ± 2.7) per match. According to the same authors, the dive is associated to the lateral save and situations of maximum intensity in which the goalkeeper performs a parry or a fly. Diving motion was analysed in four goalkeepers and it was found that the more skilled players dived faster (4 m·s^-1 as opposed to 3 m·s^-1) and more directly to the ball (Suzuki et al., 1987). The kinetic and kinematic characteristics of goalkeeper making diving saves showed that asymmetries exist in the movement patterns of goalkeepers according to the preferred or non-preferred side due to over rotational differences in the transverse plane (Spratford et al., 2009).

Hardly any information regarding goalkeeper-specific skills can be found with the exception of a recent study of Knoop et al. (2013) designed to evaluate the reaction and action speed test (RAS) among thirty-four German goalkeepers of different age groups and competitive levels. Although the results of the RAS test have successfully differenced the first goalkeepers and their substitutes, the instrumental apparatus used in this study (Knoop et al., 2013) is not easily accessible to coaches and trainers. Therefore, simpler protocols are needed to evaluate the particular technical skills involved in goalkeeping.

The overall purpose of the current research was to develop and evaluate two soccer-specific tests designed to examine goalkeeper-specific technique. The study was divided in two parts: 1) to evaluate the reproducibility of the Sprint-Keeper Test and the Lateral Shuffle-Keeper Test; and 2) to examine the construct validity of the applied tests. The assessment procedure included measures of test-retest reliability. Validity was ascertained by comparing two groups of young soccer goalkeepers of different competitive levels, hypothesizing that elite goalkeepers would perform better.

A total sample of sixty-six young male goalkeepers, all Caucasians, participated in the current research (Table 1). In the first part of this study, twenty-six goalkeepers (chronological age: 14.49 ± 2.52 years; accumulated soccer training: 5.62 ± 2.42 years; weekly volume of training: 5.8 ± 1.6 hours) completed the goalkeeper-specific tests on two separate occasions to determine test-retest reliability, with a week of interval between tests. In the second part, a subsample of eighteen elite goalkeepers (chronological age: 13.81 ± 1.81 years; accumulated soccer training: 6.00 ± 1.82 years; weekly volume of training: 6.1 1.5 hours) and twenty-two non-elite goalkeepers (chronological age: 15.04 ± 1.43 years; accumulated soccer training: 6.64 ± 2.44 years; weekly volume of training: 5.8 ± 1.3 hours) was used to examine the construct validity. Elite goalkeepers belonged to the youth department of two professional clubs and played at a national level, while non-elite players were part of amateur clubs and competed at a regional level. Chronological age (CA) was calculated to the nearest 0.01 year by subtracting birth date from date of testing. Soccer experience, i.e. accumulated soccer training, and number of hours per week devoted to soccer training and preferred diving side were obtained by interview. None of the subjects reported recent injuries.

Clubs and coaches were contacted and fully informed about the nature of the study and the procedures involving data recording. All subjects received a complete explanation about the testing procedures and that they could withdraw from the study at any time before giving their verbal consent to participate. The study received ethical approval from the Scientific Committee of the University of Coimbra taking into account the standards for sports medicine and the recommendations of the Declaration of Helsinki.

The protocols for the specific tests for soccer goalkeepers proposed and used in this study included typical movements of goalkeeping (sprint, lateral shuffle, change of direction and dive) that better mimic a game situation. Therefore, an acceptable ecological validity can be assumed. Besides, due to their simplicity and a minimum of required equipment, the testing procedures were designed to be easily available to coaches and trainers. The final version of the applied tests resulted from an original straight sprint and diving protocol. Adaptations occurred during three experimental sessions with an independent sample of six goalkeepers until the proposed tests emerged to assess goalkeeper-specific diving technique.

All field protocols were performed between 15:30 and 19:30 hours, on artificial grass with the players equipped with appropriate goalkeeper wear and soccer boots. A standardized 10 minutes warm up, which included jogging, a series of increasing intensity sprints, dynamic stretching and specific goalkeeper ball drills, was performed before the commencement. No static stretching exercises were allowed before any test (Fletcher and Monte-Colombo, 2010). Afterwards, a familiarization try-out preceded the two trials. Each subject was instructed and verbally encouraged to give their maximal effort during all trials. The subjects had 2 or more minutes of rest between 2 consecutive trials. Experimental conditions were controlled for air temperature (14 – 23ºC) and relative humidity (30 – 67%).

The subjects performed two vertical jump protocols. The first jump test consisted in the standardized counter movement jump (CMJ) during which the subjects were asked to keep their hands in their hips, to maintain their body vertical throughout the jump, and to land with their knees fully extended. The second vertical jump test was a free counter movement, jump during which the players freely swing the arms (CMJ-free arms). The vertical jump performances were evaluated by means of an optical acquisition system (Optojump, Microgate, Bolzano, Italy), developed to measure with 10^-3-second precision all flying and ground contact times. The Optojump photocells are placed at 6 mm from the ground and are triggered by the feet of the subject at the instant of take-off and are stopped at the instant of contact on landing. Then calculations of the height of the jump were made (Komi and Bosco, 1978).

Acceleration was evaluated using two sprint tests, involving straight sprinting of 5-m and 10-m as fast as possible from a standing start position. Time was recorded using a system of dual infrared reflex photoelectric cells (Polifemo; Microgate). Players began from a standing start, with the front foot 0.5 m from the first timing gate. The best result of the two trials for both vertical jump and acceleration were retained for the statistical analysis.

The first protocol was proposed to examine diving technique, involving moving as fast as possible in the direction of a stationary ball after performing a change of direction, in a total distance of approximately 10 m. The Sprint-Keeper Test (S-Keeper) was composed by the following successive phases: a) accelerating from a static standing position; b) sprinting in a straight direction; c) turning the cone while performing a change of direction and finally d) diving to catch a stationary ball. The subject who failed to catch the ball with both hands had to repeat the test. Two electronic timing gates (Polifemo; Microgate) were set up with the first gate at the start line (0 m), and the second one at a distance of 7 m to the side at a 45º angle to the marked cone, placed at a distance of 3 m in front of the starting point, according to the schematic representation in Figure 1 – panel A. The first pair of the electronic timing system sensors mounted on tripods was set approximately 75 cm above the floor and positioned 4 m apart, while the last pair was set approximately 10 cm above the floor. The ball centre was placed at a distance of 11 cm from the beam and at 8 m from the end line. S-Keeper performance was assessed for both sides and two trials were completed for each side. The best result of the two trials for each side was retained.

A second protocol was proposed to assess the diving technique, involving three changes of direction and two forms of displacement: frontward and lateral shuffle, in a total distance of approximately 11 m. The Lateral Shuffle-Keeper Test (LS-Keeper) was composed by the following successive phases: a) accelerating from a standing position; b) sprinting in a straight direction toward the cone A placed 3 m in front of the starting point; c) facing forward and without crossing feet, shuffling (lateral) toward the cone B (2 m) and then back to the cone A, always giving the inside to the cones; d) diving to catch a stationary ball. In case, during the execution of a trial, the goalkeeper was crossing one foot in front of the other or failing to catch the ball with both hands, the trial was not considered and the subject invited to repeat it. Two electronic timing gates (Polifemo; Microgate) were set up with the first gate at the start line (0 m) and the second one at a distance of 2 m to the side from cone A (placed 3 m in front of the starting point) and 5 m to the front, as shown in Figure 1 – panel B. The first pair of the electronic timing system sensors mounted on tripods was set on the start line approximately 75 cm above the floor and positioned 4 m apart, while the second pair was set approximately 10 cm above the floor. The ball centre was placed at a distance of 11 cm from the beam and at 8 m from the end line. LS-Keeper performance was assessed for both sides and two trials were completed for each side. The best result of the two trials for each side was retained for the statistical analysis.

Descriptive statistics for CA, accumulated soccer training, weekly volume of training, acceleration, vertical jump, S-Keeper and LS-Keeper were calculated for the total sample. Reliability refers to the reproducibility of a measure or variable in repeated trials on the same subjects (Hopkins, 2000). Systematic bias between repeated measures was assessed using the paired samples T-test and effect sizes estimated (Rosnow and Rosenthal, 1996). A selection of statistical methods was completed to assess random error. Relative reliability was determined using Pearson’s coefficients of correlation (r), reliability coefficients (R) (Mueller and Martorell, 1988), and intra-class correlation coefficient (ICC). Absolute reliability was determined using technical error of measurement (TEM) and coefficients of variation (%CV). The Bland-Altman procedures (Bland and Altman, 1986) were also conducted to determine limits of agreement (LOA) between sessions. Limits of agreement can be seen as tolerance intervals, and represent the test-retest differences for 95% of a population (Atkinson and Nevill, 1998). Pearson’s correlations between the means and differences of two trial sessions with accumulated soccer training, weekly volume of training, acceleration, vertical jump and goalkeeper-specific technique were calculated for both protocols according to diving side, after controlling for CA.

Construct validity refers to the degree in which a protocol measures an hypothetical construct and it can be measured by comparing two different groups of subjects with different abilities (Currell and Jeukendrup, 2008). So, comparisons between elite and non-elite goalkeepers were performed after controlling for CA. Coefficients were interpreted as follows: trivial (r < 0.1), small (0.1 < r < 0.3), moderate (0.3 < r < 0.5), large (0.5 < r < 0.7), very large (0.7 < r < 0.9), nearly perfect (r > 0.9) and perfect (r = 1) (Hopkins et al., 2009). The smallest worthwhile difference (SWD) was determined using the Cohen’s d effect size, representing the magnitude of improvement in a variable as a function of the between-subject standard deviation (SWD = 0.2 x between-subject standard deviation of young elite and non-elite goalkeepers) (Impellizzeri and Marcora, 2009; Impellizzeri et al., 2008). The true effect was considered unclear whenever the chance of benefit and harm were both ≥5%. Magnitude-based inferences about effects were qualitatively determined by the following thresholds: <0.5%: most unlikely; 0.5–5%: very unlikely; 5–25%: unlikely; 25–75%: possibly; 75–95%: likely, 95–99.5%: very likely; >99.5%: most likely (Hopkins et al., 2009). Statistical significance was set at p < 0.05 and all analyses were carried out using the Statistical Package for the Social Sciences for Windows (SPSS v.22.0, Chicago, IL, USA).

Mean results between the two trial sessions were similar for both tests when executed to the left side (Table 2). However, repeated measures for S-Keeper and LS-Keeper were significantly lower in the second trial session when performed to the right side (p = 0.010; p = 0.000). These differences were considered to be moderate (ES = 0.489) and large (ES = 0.630), respectively.

Reliability statistics for the goalkeeper-specific tests are presented in Table 3 and the Bland-Altman plots illustrated in Figure 2. Bivariate correlations between repeated measures of S-Keeper (r = 0.883; r = 0.876) and LS-Keeper (r = 0.912; r = 0.835) were high and significant. Test-retest analyses are quite similar for the protocols and side variation: S-Keeper right (TEM = 0.09; R = 0.90; ICC = 0.937; LOA = -0.28 to 0.16), S-Keeper left (TEM = 0.09; R = 0.89; ICC = 0.922; LOA = -0.23 to 0.29), LS-Keeper right (TEM = 0.19; R = 0.90; ICC = 0.950; LOA = -0.58 to 0.24) and LS-Keeper left (TEM = 0.20; R = 0.88; ICC = 0.908; LOA = -0.61 to 0.47). Data also indicated a within-subject variance of 4.18%, 4.37%, 3.97% and 4.16%, respectively for test protocol and side.

Mean performance and differences between repeated measures in goalkeeper-specific tests were verified according to diving side in Table 4. A weak to strong association with acceleration was presented. Also, a weak correlation was generally noted between accumulated soccer training and the means and differences of two trial sessions, particularly when performed for the left side. No linear association was observed between goalkeeper-specific tests and vertical jump.

When adjusted for the cofounder factor (CA), elite goalkeepers were estimated to perform better in all variables and to present lower asymmetries (Table 5). Multivariate analysis of covariance noted a significant effect of competitive level for the left side in both protocols (F = 6.111, p = 0.018; F = 5.322, p = 0.027). Nevertheless, small differences were observed among goalkeepers in the S-Keeper (d = 0.287; d = 0.219), while competitive level had a trivial (d = 0.057) and moderate (d = 0.329) effect in the LS-Keeper. The adjusted mean differences were respectively 0.024 s and 0.114 s, resulting in unclear to most likely trivial probabilities that reflect the uncertainty in the true value. The critical values in the LS-Keeper were 0.005 s (K0.11%) and 0.029 s (00.62%), after adjusted mean differences 0.136 s and 0.313 s were estimated, reflecting likely to most likely trivial chances of substantial differences between elite and non-elite goalkeepers.

The present research evaluated the reproducibility and validity of two new goalkeeper-specific tests: the S-Keeper and the LS-Keeper. The main findings regarding relative reliability indicated a high correlation between trial sessions, and high reliability for the applied tests (i.e. coefficient of reliability and ICC). When comparing the results of our study with previous literature assessing goalkeepers during specific test (Knoop et al., 2013), the S-Keeper and the LS-Keeper revealed higher values for ICC and generally larger bias. The absolute reliability statistics of our study suggested relatively little within-subject variation and fall within the general recommendations regarding coefficients of variation (<5%) in time trial protocols (Currell and Jeukendrup, 2008). Unlike previous research (Knoop et al., 2013), the mean and difference of two trial sessions in the S-Keeper and LS-Keeper had a stronger association to acceleration rather than to vertical jump performance. These differences can be partially explained by the physical nature of the RAS test (Knoop et al., 2013), where subjects were instructed to react upon an optic signal by deflecting a ball placed in one of the four angles of the goal, as opposed to our study where goalkeepers were instructed to dive to a stationary ball. A stronger relationship between diving to low balls and horizontal jump or lateral jump can be expected.

The use of paired samples T-test was able to detect a systematic bias in both protocols, particularly when goalkeepers dived on their right side. This tendency for performance in retest to be better than the prior test could suggest a general learning or fatigue effects of the tests (Atkinson and Nevill, 1998). However, this trend was curiously verified in the side that 53.5% of the subjects reported as being their preferred diving one, while a 7.7% of them referred it was indifferent to dive to the right or left side. It is possible that an increased performance between sessions was mostly related with individual’s technical proficiency in the diving action, rather than acceleration and vertical jump. In practical terms, this means that in our study goalkeepers had an enhanced efficiency in the ability to throw the body as fast and further as possible to catch the ball on their preferred side. The potential effect of the diving direction on the movement patterns of elite goalkeepers has been previously investigated (Spratford et al., 2009). The number of relationships exhibited by the thorax, pelvis, and hip kinematics, peak joint moments and centre of mass indicated that the critical time period of the dive occurred at or before the initiation phase as greater lateral rotation of the pelvis and thorax was already evident at this point for the non-preferred side. It is therefore reasonable to think that measurement errors between sessions might be more related to biological or mechanical variation (i.e. random error) (Atkinson and Nevill, 1998).

Performance in diving saves is influenced by several components (e.g. acceleration, deceleration, jumping, change of direction, side preference and diving movement), resulting in a combination of physiological, metabolic, biomechanical and morphological aspects. For instance, the technical execution during a change of direction could be conditioned by the players’ chronological age and competitive level (Condello et al., 2013). In fact, the capability of an individual to complete a relative short ground contact time and generate force in a short period of time, as well as the leg power generated during stretch-shortening cycle, could be important factors to perform rapid changes of direction (Haj-Sassi et al., 2011; Young et al., 2002). In our study the dominant leg assessment was not considered but asymmetry between lower limbs has been shown to influence performance in single leg jumps (Sugiyama et al., 2014). A variety of training techniques is therefore required in order to optimize the overall performance in diving saves, particularly by enhancing the displacement before to perform a technical action, the take-off movement and by reducing asymmetries between diving sides (Condello et al., 2013; Haj-Sassi et al., 2011; Mendez-Villanueva et al., 2011; Miyaguchi and Demura, 2010; Young et al., 2002).

In the second part of our study the construct validity was assessed by comparing the goalkeeper-specific technique in two different groups of subjects. After controlling for their CA, the elite goalkeepers groups resulted to perform significantly better to the left side in both the protocols applied in this study. Nevertheless, the competitive level had a trivial to moderate effect on the goalkeeper-specific technique. Similarly to the results found in our study, differences were observed in the RAS test proposed by Knoop et al. (2013) for both the first and substitute goalkeepers for the left side. In this regard, authors presented a possible relationship between right-handed goalkeepers and movement characteristics when diving to the right corner, as they might be more self-confident, precise and produce more acceleration. Moreover, a previous study to that of Knoop et al. (2013) suggested that in skilled motor activities a right-leg dominance may be found, although no marked lateral dominance in the take-off leg was shown on isokinetic strength of lower limbs in 27 young men who exercised regularly (Miyaguchi and Demura, 2010). Though it can be considered speculative, once again our results suggest that asymmetries related to the diving side might be influenced by individual jumping performances and diving technique (Schmitt et al., 2010). Accumulated soccer training or training volume may also have an influence, since 52.5% of this subsample claimed the right side to be their preferred, while 22.5% stated no side preference.

A diving save is an action that goalkeepers can perform according to the contextual dimension of the defensive moment, such as shooting, crossing, through ball and 1 vs. 1. Indeed, while performance in skilful movements is one of the most important aspects in goalkeeping, the tests proposed in the current study with the aim to assess goalkeeper’s technique tended to be physiological in nature (Currell and Jeukendrup, 2008). In this regard, the distinction between “skill” and “technique” has been highlighted (Ali, 2011). Skill involves the ability to select and perform efficient and effective movement patterns as determined by the contextual demands, whereas technique refers to the ability to bring about pre-determined results using a good pattern of movements. Consequently, a higher discriminative validity should include more complex protocols, involving anticipatory perceptual-motor behaviours (Knoop et al., 2013; Savelsbergh et al., 2002), with the risk to increase random error.

For team sports, in which there is no linear relationship between testing measures and the performance observed during a game, the smallest worthwhile difference (0.2 of the between-participants standard deviation) represents a sport-specific value beyond which a difference is likely to be important to detect in practical terms (Currell and Jeukendrup, 2008; Hopkins, 2000). When examining the overall performance of all the forty subjects evaluated in the second part of our study (Table 5), it would appear that those goalkeepers who were considered to possess superior goalkeeping abilities were performing better in both protocols – high construct validity (Knoop et al., 2013). Nevertheless, chances of substantial differences between elite and non-elite goalkeepers are most likely trivial. It is important to consider the influence of several components of the S-Keeper and LS-Keeper towards the overall performance, which suggest a kind of caution to evaluate the goalkeepers only based on the time to perform a predetermined technical skill.

The effectiveness of a long-term training program requires the goalkeeper to excel in a wide range of physical, technical and tactical aspects, providing appropriate game-related stimulus on short-term muscle power and perceptual-cognitive skills. Although analytical forms might be appropriate to introduce and consolidate technical contents in the period before middle adolescence, as it seems to be optimal in the technical specialization (Rebelo-Goncalves et al., 2015), it is desirable that the training methodologies emphasize the development of goalkeeping skills, in a balanced combination with technical learning aspects. The design of goalkeeper-specific activities must consider that successful goalkeeping actions during a game are measured as an effective result of physical and technical responses under temporal constraints. Finally, goalkeepers should ultimately be prepared for anticipating the pathway and direction of the kicked ball (Lidor et al., 2012) and to produce effective actions in response to variation in visual information with the aim of decreasing movement time (Dicks et al., 2011).

The current research supports a better understanding regarding performance-related characterization of soccer goalkeepers through the development and evaluation of two new goalkeeper-specific tests: the S-Keeper and the LS-Keeper. Both protocols showed high reliability and presented sufficient validity when comparing goalkeepers by competitive level. Nevertheless, assessing diving technique through a predetermined action in direction to a static ball placed on the ground compromises an individual’s ability to respond to a game-related stimulus. Applied research should incorporate anticipatory perceptual-motor behaviours in order to achieve a better ecological validity of the actions performed by goalkeepers (Lidor et al., 2012; Savelsbergh et al., 2002). Also, the potential influence of the variability associated to anthropometrical factors such as stature and the proportionality trunk/limbs was not considered in the present study. The assessment of diving technique can inform coaches and goalkeepers about good movement patterns, take-off movements and possible asymmetries between diving sides.